Skip to main content

DocumentDB

Summary​

This document covers the information to gather from AWS and DocumentDB in order to configure a Qarbine data service. The data service will use the Qarbine MongoDB API driver. You can define multiple data services that access the same DocumentDB endpoint though with varying credentials. Once a data service is defined, you can manage which Qarbine principals have access to it and its associated DocumentDB data. A Qarbine administrator has visibility to all data services.

Overview​

AWS DocumentDB is a fully managed document database service that offers some MongoDB API compatibility. It supports MongoDB versions 3.6, 4.0, and 5.0 APIs. While Amazon DocumentDB aims to be compatible with MongoDB, it does not support every MongoDB 5.0 feature. It provides a semi-MongoDB-compatible environment that can be used with existing MongoDB applications and tools, though it may not fully support all MongoDB features.

See this page for more information on compatibility
https://docs.aws.amazon.com/documentdb/latest/developerguide/compatibility.html

For detailed compatibility information and functional differences, it’s recommended to consult the official AWS documentation at https://aws.amazon.com/documentdb/.

Important Network Considerations​

Per AWS, “Amazon DocumentDB is virtual private cloud (VPC)-only and does not currently support public endpoints.” For details see the information on this page,
https://docs.aws.amazon.com/documentdb/latest/developerguide/troubleshooting.connecting.html

Remember that network access rules may apply as well depending on your VPC topology. If you are attempting to connect to your Amazon DocumentDB from outside the VPC in which your cluster resides, see Connecting to an Amazon DocumentDB cluster from outside an AWS VPC at https://docs.aws.amazon.com/documentdb/latest/developerguide/connect-from-outside-a-vpc.html

DocumentDB Configuration​

Qarbine uses its MongoDB driver to interact with DocumentDB. Refer to that guide for general information on using the MongoDB API driver. The information to be gathered for DocumentDB interaction includes:

  • TLS status,
  • user and password pair, and
  • DocumentDB cluster endpoint.

Navigate to the DocumentDB cluster page https://console.aws.amazon.com/docdb

Confirm the cluster is running.

First, determine whether TLS is used or not from the information provided. Click on your cluster of interest.

  

TLS is enabled by default for new Amazon DocumentDB clusters. However, you can disable it. For more information, see Managing Amazon DocumentDB cluster TLS settings at https://docs.aws.amazon.com/documentdb/latest/developerguide/security.encryption.ssl.html#security.encryption.ssl.managing

The TLS setting value can be seen within the parameter groups area.
Click on

  

Click on your cluster’s associated group.

  

Locate the parameter highlighted below.

  

Remember the value for this setting. It is used below.

Next, locate the endpoint. Navigate to the cluster listing page.

One way of getting there is by clicking in the left hand navigation panel

  

Then click on the cluster of interest.

  

Click on the Configuration tab as shown below.

  

  

Copy the endpoint text to a scratch pad area.

For more information on finding your documentDB cluster’s endpoints see
https://docs.aws.amazon.com/documentdb/latest/developerguide/db-cluster-endpoints-find.html

Details on connecting to DocumentDB from node.js (which Qarbine uses) can be found at
https://docs.aws.amazon.com/documentdb/latest/developerguide/connect_programmatically.html

On that page select the highlighted tab shown in the section matching your TLS setting.

  

The important code snippet is highlighted below.

  

The effective MongoDB connection string in this example is

mongodb://<sample-user>:<password>@YOUR_END_POINT:27017/sample-database?replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false

In Qarbine, you will replace the user, password, and cluster pieces to format the data service’s “server template” value. The user should be limited to read-only permissions.

The global-bundle.pem file referenced in the code above has the public key for Amazon DocumentDB. If necessary, it can be added as a server option for the Qarbine data service. This is discussed below.

Qarbine Configuration​

Compute Node Preparation​

Determine which compute node service endpoint you want to run this data access from. That URL will go into the Data Service’s Compute URL field. Its form is “https://domain:port/dispatch”. A sample is shown below.

  

The port number corresponds to a named service endpoint configured on the given target host. For example, the primary compute node usually is set to have a ‘main’ service. That service’s configuration is defined in the ˜./qarbine.service/config/service.main.json file. Inside that file the following driver entry is required

"drivers" :[
. . .
"./driver/mongoDriver.js"
]

The relevant configuration file name for non primary (main) Qarbine compute nodes is service.NAME.json. Remember to have well formed JSON syntax or a startup error is likely to occur. If you end up adding that entry then restart the service via the general command line syntax

pm2 restart <service>

For example,

pm2 restart main

or simply

pm2 restart all

Data Service Definition​

Open the Administration Tool.

Navigate to the Data Services tab.

  

The above example corresponds to the data service values shown below.

  

  

If TLS is enabled the server options are

  

tlsCAFile="./config/aws-global-bundle.pem"

Qarbine already has the AWS global PEM file in the ˜/qarbine.service/config folder with the file name of aws-global-bundle.pem. There is no need to install it manually on the Qarbine server.

If TLS is not enabled the server options are

  


useNewParser = true

Optionally enter the starting database in the provided entry field.

Next, test the settings by clicking on the icon noted below.

  

You should see the following dialog

  

The “5.0.0” corresponds to the AWS DocumentDB version in the DocumentDB console.

  

Save the Data Service by clicking on the image highlighted below.

  

The data service will be known at the next log on time.

Further information on configuring the MongoDB driver can be found in the documentation noted below.

  

Troubleshooting​

The following AWS DocumentDB page provides discussion on areas to look into regarding connectivity.
https://docs.aws.amazon.com/documentdb/latest/developerguide/troubleshooting.connecting.html

Tutorial Data​

Amazon provides sample datasets for DocumentDB that can be imported using the mongoimport tool. Sample movies JSON data can be found at
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/samples/moviedata.zip

You can import moviedata.json into the movies collection in the samples database. Remember you must either be on the EC2 instance hosting the database, on an instance in the same VPC, or have configured access to the DocumentDB database from your network location.

The format of the import command when using TSL is

mongoimport --db samples --collection movies -h YOUR_ENDPPOINT -u USER -p PASSWORD --tsl --tslCAFile global-bundle.pem --jsonArray --file moviedata.json

The command line to access the database from the MongoDB shell is

mongo -h YOUR_ENDPPOINT -u USER -p PASSWORD --tsl --tslCAFile global-bundle.pem

The format of the import command when using legacy SSL is

mongoimport --db samples --collection movies -h YOUR_ENDPPOINT -u USER -p PASSWORD --ssl --sslCAFile global-bundle.pem --jsonArray --file moviedata.json

The command line to access the database from the MongoDB shell is

mongo -h YOUR_ENDPPOINT -u USER -p PASSWORD --ssl --sslCAFile global-bundle.pem

For more information see the page at
https://aws.amazon.com/blogs/database/get-started-with-the-amazon-documentdb-jdbc-driver